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ABSTRACT 

We  give  an  algorithm  for  finding  the  kth  smallest  item  in  a  set  of  n  items,  running  in  parallel  time 
0((loglogn)-)  on  0(n)  processors  in  Valiant's  comparison  model. 
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1.  Introduction 

In  the  cx)raparison  model  of  computing  we  only  count  the  time  taken  to  carry  out  comparis- 
ons. We  ignore  time  spent  in  communications,  in  determining  which  comparisons  to  do  next,  and 
any  other  side  computations.  TTius  lower  bounds  in  such  a  model  provide  lower  bounds  in  a  more 
realistic  model.  Of  course,  upper  bounds  need  not  tr<mslate  to  a  more  re;ilistic  model;  however, 
by  concentrating  on  one  aspect  of  the  computation  we  learn  which  is  the  difficult  part.  The  model 
was  first  used  for  parallel  computations  by  Viiliant  [Vj.  He  obtained  upper  bounds,  in  the  form  of 
tradeoffs  between  the  number  of  processors  and  the  running  time,  for  computing  tlie  maximum, 
for  merging,  and  for  sorting.  He  also  gave  a  matching  lower  bound  for  finding  the  maximum, 
which  is  also  a  lower  bound  for  finding  the  kth  smallest  item.  With  0{n)  processors  the  lower 
bound  is  O(loglogn)  time. 

Subsequently,  Megiddo  [M]  has  shown  that  fast  parallel  algorithms  can  aid  in  the  design  of 
efficient  sequential  algorithms.  In  the  several  examples  he  gives,  the  communication  costs  of  the 
parallel  algorithms  can  be  ignored;  that  is,  the  important  criteria  is  that  they  be  fast  in  various  sim- 
plified models.  Recently  [CSY]  discovered  a  sequential  algorithm  which  uses  a  parallel  median 
algorithm.  Their  algorithm  partitions  n  points  in  the  plane  into  4  equal  sized  sets  by  finding  two 
appropriate  straight  lines.  It  has  a  running  time  of  0(n\ognT(n)),  where  T(n)  is  the  running  time 
of  a  median  algorithm  in  Valiant's  model.  This  ;rovides  further  motivation  for  studying  algo- 
rithms in  this  model. 

2.  The  Algorithm 

Our  algorithm  for  finding  the  kth  smallest  item  is  based  on  the  following  paradigm.  We  find 
two  approximations  to  the  kth  smallest  item  e;  one  guaranteed  to  be  no  larger  than  e,  one 

guaranteed  to  be  no  smaller  than  e.  We  use  these  approximations  to  p;u"tilion  the  set;  we  term 
the  subset  consisting  of  those  items  between  the  two  approximations  the  working  set.  The  working 
set  holds  the  item  e,  so  eilhcr  we  apply  tiie  algorillim  recursively  to  the  v/orldng  set,  or  if  the 


working  set  is  small  enough  we  sort  it  and  find  the  desired  item  directly. 

First,  we  describe  the  properties  of  the  approximation  algorithm.  Given  a  set  5  of  m  items, 
and  given  n  processors,  n^m,  it  finds  an  approximation  to  the  kth  smallest  item  in  time 
(9(max{l,loglogn-loglog(/i/m)}).    Further,  the  larger  n/m,  the  better  the  approximation.    K  the 

item  returned  is  actually  the  jth  smallest  item,  then  [j-k\  <  —  (lemma  3,  below).    For 

m-<n^'~,  we  sort  the  items,  and  find  the  kth  smallest  item,  exactly,  in  time  0(1). 

Let  us  consider  the  running  time  of  the  median  algorithm  on  n  items,  given  n  processors. 
We  define  a  stage  of  the  median  algorithm  to  be  the  sequence  of  steps  resulting  in  a  partition  of 
the  set.  That  is,  the  finding  of  two  approximations  to  the  kth  smallest  element,  each  taking  paral- 
lel time  OQoglogn)  time,  and  the  resultant  partitioning  of  the  set,  taking  time  0(1).  We  show  the 
median  algorithm  has  only  O(loglogn)  stages,  resulting  in  a  running  lime  of  0((loglogn)-). 

Lemma  1:  Suppose  the  median  algorithm  operates  on  n  items  using  n  processors.  At  the  end  of 
the  first  stage  the  working  set  is  bounded  in  size  by  n/2,  at  the  end  of  the  second  stage  by  n/4,  at 
the  end  of  the  third  stage  by  n/16,  and  for  i^2,  at  the  end  of  the  (2i)th  stage  by  n/2-''^  ',  and  at 
the  end  of  the  2i+lst  stage  by  n/2^(^-'''*^''\ 

Proof:  For  the  first  stage  we  know  each  approximation  is  in  error  by  at  most  n/4(n/n)^'^  —  n/4;  so 
the  size  of  the  working  set  is  at  most  n/2.    For  the  second  stage,  the  size  is  boimded  by 


— —  <  n/4.    For  the  third  stage,  the  size  is  bounded  by  2-—^ 

4.2-2  ^  4.4I- 


2-    "^  ,^  <  n/4.    For  the  third  stage,  the  size  is  bounded  by  2-    "        =  n/16.    For  the  fourth 


stage,  the  size  is  bounded  by  2-— <  n/2^  -  — - — .    Induction  handles  the  remaining  cases. 


\ 
Corollary:  The  median  algorithm  on  n  items  with  n  processors  takes  O(loglogn)  stages. 

Proof:  Immediate,  n 


It  remains  to  describe  the  approximation  algorithm.  We  show  how  to  obtain  an  approxima- 
tion smaller  than  the  jth  smallest  item;  finding  a  larger  approximation  uses  an  analogous  method. 
Suppose  we  wish  to  find  the  jth  smallest  item  in  a  set  5  of  m  items,  using  n  processors,  with 
m<n.  By  slowing  down  tlie  processors  by  a  factor  of  4096,  we  can  act  as  if  we  had  A^  =  4096n 
processors;  the  restriction  becomes  4096/n  ■<  N.  li  m~  <  8/V,  we  sort  the  set  S  in  0{\)  time  and 
find  the  jth  smallest  item  directly.  Othenvise  let  r  =  \{N/nif''^.  We  divide  the  m  items  into 
short  sets  of  p-  items  each;  the  leftover  items  form  another  short  set.  Let  s  be  the  number  of 
short  sets;  s  =  [m/r^].  We  sort  each  short  set,  which  we  can  do  in  0(1)  parallel  time  with  the  n 
processors.  We  form  the  set  T  consisting  of  the  {ri)th  items,  for  i-\,2,...,r,  for  each  of  the 
sorted  short  sets.   T  has  size  bounded  by  \mJr^. 

Lemma  2:  For  ;'  >  2m/ r,  the  {[i/r\  -  s)th  smallest  item  in  T  is  no  larger  than  the  jth  smallest 
item  in  S,  and  is  at  least  the  {\j/r\r  -  sr)th  smallest  item  in  5. 

Proof:  The  kth  smallest  item  in  7  is  at  most  the  {kr  +  s{r-\))th  smallest  item  in  5;  the  first  claim 
follows. 

Suppose  y  is  tlie  Ith  smallest  item  from  the  short  set  V  added  to  T;  and  suppose  x  is  the 
l-\st  smallest  (if  /=  1,  let  j:  be  an  imaginary  Of/i  item).  With  item  y  we  associate  those  items  in  V 
lying  between  x  and  y,  plus  y  itself.  We  note  those  items  associated  with  an  item  y  cannot  be 
larger  than  v;  r  items  are  associated  with  y.  We  deduce  the  kth  smallest  item  in  7  is  at  least  the 
{kr)th  smallest  item  in  5.   The  second  claim  follows,  a 

The  approximation  algorithm  for  the  jth  item  proceeds  as  follows.  Let  t  =  \j/r\  -  s.  E 
j  ^  8nj/r  the  smaller  approximation  to  the  jth  smallest  item  is  the  (imaginary)  0th  item  (i.e.  -0°). 
Otherwise  it  is  the  smaller  approximation  to  the  tth  smallest  item  in  T,  obtained  recursively.  Since 
this  is  at  most  the  tth  smallest  item  in  T,  it  is  at  most  tlxcjth  smallest  item  in  S  (by  lemma  2). 

Lemma  3:  Tlie  error  Ln  finding  the  approximation  to  the  jth  smallest  item  in  5  in  bounded  by  Sm/r 
(<ni/4(n/w)"-),  iiN^409t,m. 


o  • 
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Proof:  We  prove  the  result  for  each  value  of  N  by  induction  on  the  size  of  m.  For  mr<SN  the 
error  is  zero.  So  suppose  m-^SA'.  The  enor  is  a  combination  of  two  errors:  the  error  in  comput- 
ing an  approximation  to  an  item  in  T,  and  the  error  in  the  item  sought  in  T.  By  the  inductive 
hypothesis,  applied  to  the  search  in  T,  the  error  in  T  for  the  item  sought  in  T  is  bounded  by 
Slm/ryu,  where  u  =  \{N/\m/r]y'-].  So  the  item  found  is  at  least  the  i[j/r\  -  s  -  S\mJr]/u)th 
smallest  item  in  T,  which  in  turn  is  at  least  the  kth  smallest  item  in  S  where 

k  =  y/r\r  -sr-  M^  >  ^^  -  2sr  -  M^?^ 


(since  \j/r\r  >  j  -  sr).    Now  sr  -  \m/r^]r  <2m/r  and  -^^^^  <  r\m/ry'-  ^  ^^^m^vi 
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<  -^       (for      A'  >  4096m).        Hence 
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it  sj  -  ^  -  -^  s  ;  -  -^.   So  the  error  is  bounded  by  -^.  □ 
r  2r  r  r 


It  remains  to  compute  the  running  time  of  the  approximation  algorithm.  We  define  a  stage 
of  the  approximation  algorithm  to  be  the  work  done  in  obtaining  the  set  T  from  the  set  5.  A 
stage  takes  time  0(1)  to  execute.  The  approximation  algorithm  has 
0(max{l,loglogn  -  loglog(n/m)})  stages;  so  its  running  time  is  0(max{l,loglogn  -  loglog (n/m)}), 
which  is  0(\og]ogn). 

Lemma  4:  The  median  finding  algorithm  runs  in  time  0((loglogn)^)  on  n  processors. 

3.  Remarks  and  Conclusions 

We  have  given  a  deterministic  parallel  time  algorithm  for  computing  the  median,  running  in 
time  0((loglogn)'),  in  Valiant's  comparison  model.  Currently  the  best  lower  bound  is  a)(loglogn). 
Can  this  be  improved,  as  conjectured  by  Valiant,  or  alternatively  can  one  find  a  faster  parallel 
algorithm?  We  believe  it  would  be  hard  to  speed  up  the  method  we  have  given  as  it  does  not 
seem  that  the  calls  to  the  approximation  algorithm  c;ui  be  run  in  parallel. 


The  method  we  give  also  suggests  reexamining  the  linear  time  median  algorithm;  could  one 
find  a  fast  sequential  algorithm  for  finding  an  approximation  to  the  median  and  use  this  as  the 
partition  item? 

The  other  natural  question  is  whether  a  probabilistic  method  can  lead  to  a  faster  runtime. 
This  has  been  answered  by  by  Reischuk  [R],  who  gives  an  algorithm  with  expected  running  time 
0(1). 
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